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ANNUAL  TECHNICAL  REPORT:  REMINDING-BASED  LEARNING 


OBJECTIVES 

When  learning  a  new  cognitive  skill,  novices  spend  much  of  their  time  solving 
problems.  In  doing  so,  it  is  common  for  novices  to  think  back  to  an  earlier  problem  that 
the  current  problem  reminds  them  of  and  use  this  earlier  problem  to  help  solve  the 
current  problem  (e.g.,  Ross,  1984,  1987,  1989a).  This  use  of  the  earlier  problem  not  only 
affects  performance  on  the  current  problem,  but  also  provides  the  learner  with 
additional  knowledge  that  can  be  accessed  and  used  on  later  problems.  The  aim  of  this 
research  is  to  understand  the  nature  of  the  learning  that  results  from  this  use  of  earlier 
problems.  Little  is  known  about  this  crucial  source  of  learning. 

Such  within-domain  analogies  occur  frequently  during  learning.  In  the  view 
presented  in  this  work,  a  generalization  is  formed  from  making  an  analogy  between 
problems.  Rather  than  positing  a  separate  generalization  process  that  operates  upon 
completed  instances,  the  generalization  may  be  a  byproduct  of  the  analogy.  In  using  the 
earlier  problem  to  help  solve  the  current  problem,  comparisons  must  be  made  and  some 
aspects  generalized  over.  Remindings,  by  setting  up  the  analogy,  may  determine  what 
pairs  of  problems  are  compared  and,  hence,  what  generalizations  are  made.  The 
learning  comes  about  because,  while  the  noticing  might  be  based  on  a  variety  of  similar 
aspects  (including  superficial  ones)  between  the  problems,  the  comparisons  forces  the 
generalization  of  many  of  the  aspects.  My  earlier  work  (e.g.,  Ross  &  Kennedy,  1990) 
has  shown  that  the  use  of  earlier  problems  allows  novices  to  begin  to  form 
generalizations  across  problems.  Thus,  this  means  of  learning  may  be  one  way  in  which 
novices  can  begin  to  develop  more  expert-like  knowledge  structures.  However,  much 
work  remains  to  be  done  to  understand  this  learning.  The  research  I  have  been 
conducting  in  the  last  two  and  a  half  years  has  three  goals. 

The  first  goal  is  to  understand  the  nature  of  the  resulting  generalization,  the 
information  included  in  its  representation.  If  the  generalization  results  from  the 
reminding,  it  is  likely  to  be  a  conservative  one,  in  that  it  will  be  somewhat  tied  to  the 
problems  from  which  it  arises.  In  fact,  most  theories  of  learning  assume  some 
conservatism,  but  my  research  (a)  distinguishes  among  different  types  of  assumptions  of 
conservatism,  (b)  relates  these  distinctions  to  current  theories,  and  (c)  tests  these 
distinctions.  Thus,  this  work  examines  the  specificity  of  what  is  learned,  as  well  as  its 
generality. 

The  second  goal  of  this  research  is  to  examine  the  implications  of  these  ideas  for 
the  development  of  expertise.  Research  on  expertise  in  mathematics  and  physical 
science  domains  suggests  that  experts  have  problem  schemas  that  allow  them  to 
categorize  problems,  as  well  as  associated  procedures  for  solving  problems  of  that  type 
(e.g.,  Chi,  Feltovich,  &  Glaser,  1981).  Despite  the  importance  of  these  schemas,  little 
research  examines  how  they  are  learned.  A  common  idea  is  that  they  may  develop  from 
the  comparison  of  problems.  However,  this  idea  leads  to  two  questions?  One,  how  do 
people  know  which  problems  to  compare?  Two,  why  are  people  comparing  problems 
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(i.e.,  what  is  the  nature  of  this  comparison)?  The  reminding-based  learning  view 
suggests  that  people  compare  problems  when  one  makes  them  think  back  to  another  and 
they  do  so  in  order  to  use  the  earlier  one  to  solve  the  current  problem.  In  addition,  this 
view  suggests  that  people  may  develop  problem  schemas  that  are  influenced  by  the 
superficial  aspects,  because  these  aspects  are  known  to  affect  remindings.  Thus,  part  of 
this  project  is  concerned  with  the  development  of  such  problem  schemas  and  the 
possibility  that  some  schemas  may  critically  depend  upon  superficial  contents,  even  in 
experts. 

The  third  goal  of  this  project  is  to  begin  to  extend  this  work  to  more  everyday 
problem  solving  situations.  This  extension  will  not  only  allow  the  application  and  test  of 
these  ideas  in  an  important  new  setting,  it  will  also  force  the  extension  of  this  work  to 
important  situations  that  are  hard  to  experiment  with  in  more  formal  domains. 


STATUS  OF  PROJECTS 


In  this  section,  I  will  provide  some  details  of  the  status  of  the  projects  underway. 
For  each  goal,  I  will  first  give  a  brief  summary  of  the  findings  and  then  provide  further 
details.  This  report  builds  upon  the  report  submitted  in  August  of  1991. 


L  Conservatism  of  learning 

As  described  earlier,  this  project  examines  the  representation  of  the  resulting 
generalization.  The  focus  here  has  been  on  asking  how  irrelevant  aspects  may  affect  the 
access  and  use  of  the  generalization. 


Summary.  A  common  theme  across  many  learning  models  is  that  generalizations 
are  accessed  solely  by  those  features  that  are  contained  in  the  generalization  (i.e.,  the 
features  common  across  the  instances).  One  experiment  shows  that  people  may  first 
access  instances  to  then  provide  access  to  the  generalizations.  That  is,  a  manipulation 
that  affects  the  access  of  instances  improved  performance  even  though  if  the  instance 
were  being  used  directly,  performance  would  not  have  been  increased  (the  results  are 
below).  This  result,  should  it  hold  up  in  replications  and  extensions,  will  be  difficult  for 
many  current  theories  to  account  for.  In  addition,  another  project  has  been  examining 
how  the  representation  of  the  generalization  may  depend  upon  the  details  of  how  the 
earlier  problem  is  used. 

Method.  The  basic  paradigm  for  examining  the  effect  of  using  an  earlier  problem 
is  referred  to  as  the  cuing  method.  The  typical  experiment  involves  study  examples,  first 
test  problems  (with  cues  to  the  study  examples),  and  second  test  problems.  This  simple 
case  allows  us  to  isolate  the  specific  influences  on  each  test  and  to  detail  the  learning  on 
a  step-by-step  level.  More  specifically,  subjects  are  instructed  in  a  simplified  formal 
domain,  elementary  probability  theory.  They  are  given  a  short  introduction  to  some 
probability  concepts  and  then  learn  four  principles  (e.g.,  permutations).  For  each 
principle,  an  explanation  of  the  principle  with  the  appropriate  formula  is  given.  The 
subject  then  helps  to  solve  a  word  problem  that  requires  the  use  of  the  principle.  This 
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study  problem  is  presented  in  a  workbook  format,  with  the  subjects  guided  through  the 
solution.  After  learning  these  principles  (4  mins/principle),  subjects  are  given  first  test 
problems  to  solve  (3  mins).  In  many  of  the  studies,  some  or  all  of  these  first  test 
problems  include  a  cue  (e.g.,  "this  is  like  the  earlier  golf  problem"),  which  has  been 
shown  to  result  in  a  generalization  (Ross  &  Kennedy,  1990).  After  each  problem, 
feedback  is  given  and  subjects  study  the  solution.  A  second  test  is  then  given.  Of  main 
interest  is  performance  on  the  second  test  problem  as  a  function  of  the  experimental 
manipulation.  The  second  test  measure  varies  for  different  experiments,  but  for  all  but 
one  of  the  experiments  (to  be  noted)  in  this  section,  the  measure  is  how  able  subjects 
are  to  instantiate  a  provided  formula  (i.e.,  they  are  given  the  appropriate  formula  and 
have  to  fill  in  the  numbers  correctly  for  the  second  test  problem).  Earlier  studies  have 
shown  this  use  measure  to  be  very  different  from  the  ability  to  determine  the 
appropriate  formula  and  it  appears  to  be  a  function  of  subjects’  understanding  of  what 
the  variables  represent.  In  all  of  these  experiments,  principles  are  rotated  through 
conditions  and  several  test  examples  are  used  for  each  principle  to  avoid  effects  due  to 
any  particular  example. 

la.  Can  irrelevant  distinctive  aspects  provide  access  to  the  generalization?  One 
study  provides  a  test  of  how  instances  and  the  generalizations  they  give  rise  to  are 
related.  That  is,  the  proposal  provides  an  analysis  of  a  number  of  ways  in  which  the 
instances  and  the  generalization  might  be  "connected".  One  notion  of  conservatism  is 
that  the  generalization  is  still  closely  tied  to  the  instances.  In  particular,  it  is  possible 
that  even  some  superficial  aspects  that  occur  in  only  one  of  the  examples  may  still 
provide  access  to  the  generalization.  This  idea,  if  true,  is  problematic  for  many  theories 
because  they  assume  that  the  generalization  includes  only  features  that  were  common  to 
the  instances  that  gave  rise  to  the  generalization.  Thus,  if  the  superficial  aspect  occurred 
in  only  one  instance,  it  could  not  have  been  in  common  and  should  not  be  part  of  the 
generalization.  The  difficulty  with  testing  this  idea  is  that  an  experimental  manipulation 
that  helps  to  access  an  instance  may  increase  performance  because  the  instance  is  used 
rather  than  because  the  generalization  is  used.  (Similar  to  the  exemplar  accounts  of 
many  of  the  prototype  effects  in  categorization).  Thus,  what  is  needed  is  a  situation  in 
which  direct  use  of  the  instance  hurts  performance,  while  indirect  use  of  the  instance  to 
access  the  generalization  helps  performance.  That  was  the  purpose  of  this  study,  which 
was  included  in  my  first  annual  report. 

That  experiment  made  use  of  an  earlier  manipulation  (Ross,  1987,  1989b)  in 
which  object  correspondences  were  shown  to  affect  how  people  instantiated  a  formula. 
More  specifically,  if  novices  learn  about  one  principle  in  which  the  variables  are 
attached  to  certain  objects,  a  later  test  with  the  same  formula  will  often  lead  them  to  try 
to  instantiate  the  formula  by  matching  the  objects.  For  example,  if  permutations  were 
learned  with  the  number  of  objects  (n)  being  mechanics  and  the  number  of  objects 
selected  (r)  being  cars,  at  a  test  with  the  formula  and  a  word  problem  involving 
mechanics  and  cars  the  novices  would  be  likely  to  again  assign  mechanics  to  the  variable 
n  and  cars  to  r.  This  assignment  occurs  even  if  the  word  problem  has  been  changed  to 
make  the  reversed  assignment  correct.  Thus,  these  earlier  studies  use  this  manipulation 
of  reversed  object  correspondences  to  show  that  the  instantiation  of  the  formula  relies 
on  superficial  aspects.  In  fact,  the  studies  in  Ross  ( 1989b)  show  that  this  superficial 
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similarity  does  not  have  to  be  at  the  level  of  mechanics  and  cars,  but  rather  at  the  level 
of  animate  and  inanimate.  That  is,  novices  will  also  reverse  correspondences  if  the  test 
problem  has  teachers  (rather  than  mechanics)  assigned  to  classrooms  (rather  than  cars). 

The  reason  why  I  have  gone  to  this  level  of  detail  is  that  these  reversed 
correspondences  allow  a  test  of  the  issue  under  consideration  here.  In  particular, 
assume  that  we  have  used  the  cuing  paradigm  to  get  generalizations.  Then,  we 
manipulate  the  superficial  similarity  of  the  story  line  to  affect  the  likelihood  of  accessing 
a  particular  instance.  If  the  instance  being  accessed  has  reversed  correspondences  to  the 
test  problem  then  using  it  will  hurt  performance.  Thus,  if  it  helps  performance  the 
instance  must  not  have  been  used  directly,  but  rather  indirectly  to  access  the 
generalization.  The  following  table  may  help  to  make  this  concrete.  The  examples  are 
written  with  the  story  content  then,  in  parentheses  a  type  of  object  correspondence  in 
which  different  numbers  mean  unrelated  correspondences  and  R  means  reversed.  In 
both  conditions,  the  first  test  problems  were  cued  to  increase  the  probability  that  a 
generalization  was  formed  between  the  study  and  first  test  problems  (as  shown  in  Ross 
&  Kennedy,  1990).  The  subjects  received  one  of  the  two  conditions  for  each  principle, 
but  had  two  principles  in  each  condition. 

Study  example  First  Test  Second  Test  (with  formula) 

Golfers  (1)  Mechanics  (2)  Golfers  (1-R) 

Golfers  (1)  Mechanics  (2)  Dancers  (1-R) 

Consider  the  difference  between  the  two  conditions,  which  shall  be  called  the 
golfers  and  dancers.  After  the  first  test,  I  assume  that  subjects  have  some  information 
about  each  of  the  first  two  problems  and  a  generalization  formed  by  comparing  the  two 
problems  (Ross  &  Kennedy,  1990,  showed  that  this  manipulation  led  to  a 
generalization).  On  the  second  test,  the  golfers  problem  is  likely  to  make  subjects  think 
back  to  the  earlier  study  example  (as  has  been  shown  a  number  of  times  in  m)  earlier 
work).  However,  if  subjects  use  this  study  example  it  will  lead  to  poor  secjnd  test 
performance  because  the  correspondences  are  reversed.  Thus,  most  theories  would 
predict  that  golfers  will  lead  to  worse  performance  than  dancers  (if  the  study  example  is 
used)  or  equal  performance  (if  only  the  generalization  is  used).  However,  if  the  study 
example  could  be  used  to  access  the  generalization,  then  performance  in  golfers  might 
be  higher  than  in  dancers. 

In  the  experiment,  the  results  confirmed  this  last  prediction.  A  study  with  24 
subjects  each  learning  4  principles  showed  that  performance  in  the  golfers  condition  was 
.71  (i.e.,  71%  of  the  formulae  were  appropriately  instantiated)  compared  to  only  .53  in 
the  dancers  condition.  (Technically,  the  t(23)  =  1.69,  which  is  not  reliable  by  a  two- 
tailed  test.  However,  this  is  due  to  one  subject  who  showed  the  exact  opposite  effect 
and  increased  the  standard  deviation  by  25%.  I  have  examined  the  data  in  a  number  of 
ways  and  any  way  that  reduces  that  subject’s  influence  leads  to  a  highly  reliable  effect. 

In  addition,  this  effect  occurs  for  all  4  of  the  principles). 


A  parallel  experiment  tested  the  materials  to  show  that  in  fact  the  reversed 
correspondences  did  not  inadvertently  help  performance.  In  particular,  this  exact 
experiment  was  conducted  on  another  24  subjects  but  instead  of  receiving  the  first  test 
problems,  they  participated  in  a  filler  task  for  the  same  amount  of  time.  For  the  first 
experiment  effect  to  be  attributable  to  the  indirect  access  of  the  generalization,  it  is 
necessary  to  check  that  the  golfer  problem  does  not  lead  to  higher  performance  than  the 
dancers  problem  when  no  generalization  is  likely  to  be  formed.  As  expected,  the  golfers 
problem  led  to  slightly  worse  performance  than  the  dancers  problem  (.31  vs.  .36).  A 
final  experiment  replicated  the  first  study  with  the  only  change  being  a  20  minute  delay 
between  the  first  test  and  the  second  test.  Here  too  the  golfers  problem  led  to  higher 
performance,  but  the  difference  was  only  .40  vs.  .35  and  was  not  reliable. 

My  next  step  was  to  address  the  same  issue  with  a  different  design.  The  long 
discussion  of  object  correspondences  should  make  this  study  easy  to  explain.  One 
difficulty  with  the  earlier  study  is  that  it  required  the  second  test  answer  to  reverse  the 
study  problem  (with  similar  content),  so  that  the  cuing  by  content  had  to  work  against 
the  use  of  the  instance.  In  this  study,  the  object  correspondences  are  such  that  the  study 
example  correspondences  are  unrelated  to  the  second  test,  so  that  we  may  better  see  the 
access  of  the  generalization.  The  design  is  as  follows,  again  with  all  the  first  tests  cued. 


Study  example 

First  Test 

Second  Test  (with  formula) 

Golfers  (1) 

Mechanics  (1-R) 

Golfers  (2) 

Golfers  (1) 

Mechanics  (1-R) 

Dancers  (2) 

As  in  the  previous  study,  we  predict  that  the  second  test  Golfers  will  lead  to 
higher  performance  (instantiating  a  provided  formula)  than  the  Dancers  problem.  Why 
might  this  be  so?  One  possibility  is  that  the  Golfers  study  example  is  more  likely  to  be 
used  to  help  solve  the  Golfers  second  test  problem.  Another  control  experiment,  like 
the  one  done  for  the  previous  experiment  (i.e.,  without  the  first  test)  can  test  this 
explanation.  The  alternative,  and  hypothesized  explanation  is  that  the  Golfers  cover 
story  allows  the  access  of  the  generalization  through  the  study  example. 

The  results  of  this  study  are  in  the  predicted  direction,  but  not  statistically 
reliable.  The  first  24  subjects  showed  a  small  effect  of  7.9%.  I  replicated  the  study 
again  with  12  subjects,  and  again  found  a  small  unreliable  effect  of  8.3%  Although 
these  results  suggest  there  may  be  an  effect,  its  small  size  (about  half  of  what  I  usually 
work  with)  requires  far  more  subjects.  Instead,  I  am  working  on  a  more  sensitive  design 
and  also  considering  trying  a  different  domain. 

I  believe  that  these  studies  are  important  for  understanding  the  nature  of  the 
generalization  and  its  hypothesized  conservatism.  Although  this  recent  lack  of 
statistically  significant  results  requires  me  to  change  my  plans  somewhat,  the  consistency 
of  the  effect  is  at  least  suggestive  that  the  main  idea  may  be  on  track.  I  hope  that  my 
planned  changes  will  allow  me  to  continue  to  address  this  issue. 


6 


lb.  The  effect  of  problem  use  on  the  generalization.  In  addition  to  testing  the 
conservatism  of  the  generalization  in  the  way  described  above,  a  central  idea  of  the 
reminding-based  view  is  that  the  generalization  depends  upon  the  earlier  problem 
comparison.  This  selectivity  effect  has  been  investigated  by  me  in  the  context  of 
category  learning  (Ross,  Perkins,  &  Tenpenny,  1990),  because  in  formal  domains 
different  earlier  problems  will  often  lead  to  very  similar  generalizations.  However,  in 
this  study  I  examine  whether  the  details  of  how  the  problem  is  used  (rather  than  which 
problem)  affects  what  is  learned.  As  will  be  seen,  it  can  be  viewed  as  a  type  of  transfer- 
appropriate  processing  in  problem  solving. 

(i).  Problem  categorization:  The  earliest  studies  using  the  cuing  paradigm 
showed  that  cuing  on  the  first  test  led  to  improved  performance  on  the  second  test  in 
three  ways:  better  problem  solving  performance,  higher  probability  of  accessing  the 
appropriate  formula,  and  better  instantiation  of  a  provided  formula  (Ross  &  Kennedy, 
1990,  Exs.  1  and  2).  After  conducting  these  studies,  I  also  became  interested  in  problem 
categorization,  which  is  thought  to  be  a  crucial  element  in  developing  expertise.  Thus,  I 
used  the  cuing  paradigm  but  instead  of  a  second  test  problem,  subjects  were  given  12 
problems  (3  of  each  principle)  and  asked  to  categorize  them  by  principle.  That  is,  at  the 
first  test,  half  of  the  principles  were  cued  and  half  were  not.  If  cuing  affects  problem 
categorization,  then  higher  categorization  should  occur  for  the  cued  principles.  In  one 
study,  I  gave  them  formulae  to  use  as  the  categories  for  sorting  and  in  another  I  gave 
them  the  principle  names.  The  (unexpected)  result  was  that  the  cuing  had  no  effect. 

In  my  first  annual  report,  I  reported  a  study  that  investigated  one  difference 
between  the  cue  method  and  the  usual  way  in  which  people  think  of  earlier  problems. 
Usually  when  people  are  reminded  they  need  to  decide  whether  the  reminding  is 
appropriate.  That  is,  they  need  to  decide  whether  to  use  the  reminding  or  not,  since  we 
often  get  reminded  of  earlier  problems  for  inappropriate  reasons.  It  seems  to  me  that 
this  difference  may  be  crucial  because  a  determination  of  appropriateness  may  include 
deciding  whether  the  problems  are  of  the  same  "type"  so  that  one  could  be  used  to  help 
solve  another.  Thus,  such  a  determination  might  affect  what  is  learned  about  problem 
categories. 

The  study  used  the  same  procedure  and  materials  that  led  to  no  cuing  effect  and 
made  one  change.  On  the  principles  that  subjects  are  cued,  they  are  told  there  is  a 
chance  that  these  cues  are  not  correct.  Their  task  is  to  first  decide  if  the  cue  is  correct 
or  not.  They  are  then  given  feedback  on  this  response  and  given  the  correct  cue.  In 
fact,  all  the  cues  are  correct  so  they  are  given  exactly  the  same  cues  as  in  the  earlier 
study,  but  they  have  to  determine  appropriateness  before  using  the  cued  problem.  The 
results  in  this  case  showed  an  effect  of  cuing,  with  the  cued  principles  leading  to  .47  of 
the  second  test  problems  being  correctly  categorized  compared  to  .30  for  the  not  cued 
principles,  t(29)  =  2.17.  The  studies  were  conducted  at  different  times  and  have  one 
minor  procedural  difference,  so  I  am  planning  to  run  one  study  in  which  both  cuing 
conditions  can  be  compared.  (If  the  two  experiments  are  compared,  the  interaction  is 
significant,  t(68)=  2.18). 

However,  last  year,  I  replicated  this  study  twice  and  found  no  significant 


7 


difference  between  the  cued  and  uncued  conditions  with  either  type  of  cuing.  This  lack 
of  replication  leads  me  to  believe  that  the  effect  with  these  materials,  if  it  exists,  is 
small.  The  conceptual  puzzle  is  that  the  effects  of  cuing  on  problem  solving  and  formula 
instantiation  are  so  large.  I  have  taken  two  approaches  to  this  problem  in  the  last  six 
months  work. 

First,  one  possible  way  to  reconcile  these  two  findings  is  to  argue  that  the 
categorization  test  may  not  be  tapping  "true"  categorization  processes.  That  is,  if  I  am 
interested  in  how  cuing  affects  problem  categorization  during  problem  solving,  I  need  to 
engage  the  appropriate  problem  solving  processes.  The  categorization  task  is  different 
enough  from  problem  solving  that  it  may  lead  the  subjects  to  use  very  different 
processes  and  strategies.  Perhaps  cuing  would  have  a  large  effect  on  categorization 
during  problem  solving,  but  little  effect  on  problem  categorization  separate  from 
problem  solving.  To  test  this  idea  (suggested  to  me  at  a  talk),  the  subjects  are  asked  to 
solve  the  problems  on  the  final  test  (using  the  principle  names  and  associated  formulae 
provided  on  a  list)  and  then  to  circle  the  principle  used.  This  procedure  forces  them  to 
engage  in  categorization  to  problem  solve  but  (surprisingly  to  me  at  least),  it  led  to  no 
effect  of  cuing  on  the  earlier  test. 

Second,  the  materials  I  have  used  in  these  studies  were  not  designed  originally  for 
a  categorization  test  and  they  are  quite  dissimilar  across  the  different  principles.  The 
problems  were  rewritten  to  be  a  little  more  confusing  and  we  did  find  an  effect,  albeit  a 
small  marginally  reliable  one  of  .06  (these  studies  gather  far  more  observations  per 
subject  by  repeated  testing  of  each  problem  category).  This  experiment  is  still  in 
progress  and  has  been  conducted  by  my  graduate  RA,  Marty  Preslar. 

(ii)  Direct  reports  on  what  was  learned:  The  earlier  studies  showing  an  effect  of 
the  cuing  on  what  was  learned,  examined  problem  solving  performance.  In  the  last  year 
and  a  half,  I  have  conducted  two  new  studies.  In  this  work,  at  the  second  test,  the 
subjects  were  directly  queried  on  what  they  knew  about  each  principle.  A  pilot  study 
with  8  subjects  (with  responses  tape-recorded  and  transcribed)  suggested  an  interesting 
result.  The  cuing  led  to  no  better  conceptual  understanding  of  the  principle  (as 
measured  by  the  explanation  of  when  the  principle  might  be  sued  and  what  it  did). 
However,  when  the  subjects  were  asked  to  explain  the  formula,  the  cuing  condition  led 
to  considerably  better  understanding  of  what  the  variables  were  and  how  they  would  be 
instantiated.  This  is  an  interesting  result,  in  that  it  suggests  that  cuing  may  selectively 
help  the  novice  in  learning  certain  aspects  of  the  principles.  In  particular,  the  low  level 
information  about  variables  may  be  learned  so  that  later  problem  solving  performance  is 
improved.  (More  speculatively,  I  believe  that  this  result  may  give  us  some  insight  as  to 
how  novices  may  use  some  knowledge  to  bootstrap  their  learning  of  other,  more 
conceptual  knowledge.  In  this  scheme,  low-level  knowledge  about  variables  may  provide 
some  later  clues  as  to  how  the  principle  is  working  in  a  conceptual  way.) 

Recently,  my  graduate  RA  Marty  Preslar,  has  followed  up  this  study  with  a  much 
larger  one  (n=24)  in  which  we  also  asked  the  subjects  to  generate  the  formula.  We 
have  a  preliminary  analysis,  but  the  results  are  currently  being  rescored  by  a  second 
judge.  So  far  it  seems  clear  that  the  explanations  are  no  better  in  the  cuing  condition 
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(just  as  we  found  with  the  pilot  study).  However,  cuing  does  lead  to  better 
understanding  of  the  variables  (again  as  in  the  pilot  work)  and  to  better  generation  of 
the  formula. 

2.  The  development  of  problem  solving  expertise 

As  discussed  earlier,  reminding-based  learning  provides  one  perspective  on  the 
nature  of  problem  schemas  and  their  development.  The  work  already  discussed  can  be 
viewed  as  very  early  precursors  to  problem  schemas,  but  no  evidence  has  yet  been 
presented  that  in  fact  they  do  lead  to  problem  schemas. 

My  focus  has  been  on  the  specialized  schemas  that  often  include  superficial 
information  as  well.  To  study  this,  I  have  written  a  number  of  algebra  word  problems 
(e.g.,  distance,  interest,  mixture).  Each  problem  has  two  versions:  one  with  appropriate 
(typical)  contents  and  one  with  neutral  contents.  The  first  two  studies  conducted  have 
followed  up  observations  noted  in  a  chapter  a  number  of  years  ago  (Hinsley,  Hayes,  & 
Simon,  1977).  First,  we  have  protocols  of  subjects  solving  these  problems  in  order  to  ask 
how  they  might  differentially  solve  the  appropriate  and  neutral  problems.  Hinsley  et  al. 
argued  that  the  appropriate  ones  were  solved  by  schema  instantiation,  while  the  others 
were  solved  by  translating  each  sentence  to  an  equation.  This  result  is  quite  important, 
but  their  results  were  based  on  few  observations  and  no  quantitative  data  were  provided. 
However,  we  were  unable  to  replicate  this  finding,  not  because  it  is  incorrect,  but 
because  so  few  of  the  problems  were  solved  at  all  (less  than  50%),  making  it  impossible 
to  see  differences  between  conditions.  We  considered  several  training  studies,  but  opted 
for  what  I  think  will  be  a  more  fruitful  tack.  An  honors  student,  Steven  Blessing,  has 
redone  this  experiment  with  three  major  changes.  First,  we  have  used  high  math 
students  (UI  students  who  graduated  from  a  math  and  science  academy).  Second,  he 
has  included  a  set  of  inappropriate  problems  as  well,  that  is  problems  of  a  given  type 
with  the  story  contents  of  a  different  type.  Third,  we  have  examined  the  time  and 
proportion  correct.  Our  finding  is  that  the  appropriate  and  neutral  conditions  do  not 
differ,  but  they  are  both  more  accurate  and  faster  than  the  inappropriate  condition.  In 
addition,  he  has  collected  problem  solving  protocols  from  another  group  of  subjects  that 
he  is  in  the  process  of  scoring. 

In  the  second  study  originally  conducted,  other  subjects  received  the  same 
problems  one  clause  at  a  time  and  were  required  after  each  clause  to  say  what  type  of 
problem  it  is.  Hinsley  et  al.  showed  people  could  do  this  quite  readily.  Our  interest  is 
in  asking  how  this  measure  may  differ  for  appropriate  and  neutral  contents.  The 
approoriate  problems  are  categorized  much  earlier. 

We  are  currently  conducting  two  studies  to  put  these  results  altogether.  First,  the 
high  math  subjects  are  being  tested  in  the  clause  study  with  appropriate,  neutral,  and 
inappropriate  problems.  Second,  the  regular  subjects  (i.e.,  UI  students  not  from  the 
math  academy)  will  participate  in  the  timed  problem  solving  task. 

Together,  these  studies  and  the  follow-up  experiments  will  provide  important 
information  about  the  problem  schemas  and  their  use,  particularly  concerning  the 
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inclusion  and  use  of  superficial  aspects.  Our  current  hypothesis  is  that  the  usual  or 
appropriate  contents  allow  one  to  more  quickly  categorize  problems  (i.e.,  the  clause 
result),  but  that  even  neutral  problems  are  generally  categorized  correctly  by  the  end  of 
the  problem  (another  result  from  the  clause  study).  Thus,  there  is  little  accuracy  or 
latency  difference  in  problem  solving  between  these  two  conditions.  However,  the 
inappropriate  problems  are  more  difficult  to  categorize  and  the  categorization  is  often 
uncertain  even  after  the  problem  has  been  read. 

3.  Everyday  problem  solving 

In  addition  to  the  work  on  probability  theory  and  formal  domains,  I  have  been 
examining  how  remindings  may  be  used  in  less  formal  situations.  In  particular,  I  have 
been  examining  how  they  may  affect  the  simple  categories  that  are  learned.  I  view  this 
work  as  parallel  to  the  problem  solving  work,  but  it  sometimes  allows  me  to  more 
cleanly  investigate  certain  issues,  especially  selectivity  effects  (Ross,  Perkins,  & 
Tenpenny,  1990). 

Summary.  The  work  from  last  year  (with  a  student  and  summer  RA,  Tom 
Spalding)  has  extended  this  earlier  paper  to  show  (1)  that  reminding-based 
generalizations  occur  in  a  common  categorization  study  paradigm,  (2)  that  remindings 
serve  to  focus  the  learner  on  to  relevant  features,  (3)  that  these  manipulations  lead  to 
differences  in  perceived  frequency,  and  (4)  that  they  also  affect  people’s  prediction  of 
the  values  for  missing  features.  Spalding  has  also  been  able  to  show  that  these  effects 
hold  up  with  different  ways  of  getting  subjects  to  think  back  to  the  earlier  examples. 
These  results  greatly  extend  the  earlier  work  and  address  many  potential  criticisms  of  it. 
I  believe  that  this  idea  may  help  to  provide  one  means  by  which  categories  can  be 
learned  and  help  relate  such  learning  to  performance  issues.  I  have  written  a  chapter 
based  on  these  ideas  (Ross  &  Spalding,  1991),  which  I  believe  I  sent  in  earlier.  In 
addition,  Spalding  has  presented  this  work  at  MPA  and  is  in  the  process  of  writing  a 
journal  paper  on  this  work. 

3a.  Reminding-based  generalizations  in  category  learning.  In  this  study,  reported 
in  the  first  report,  we  show  that  the  manipulation  of  a  feature  that  affects  reminding 
(color),  but  which  subjects  know  will  not  be  included  on  tests,  still  affects  what  is 
leaned  about  the  category.  As  an  abstract  example,  the  letters  a,  b,  c,  d,  e,  f,  g  and  h 
stand  for  features  (e.g.,  a  might  be  "likes  gardening").  Subjects  then  learned  about 
people  from  two  categories.  The  structure  for  each  category  was  as  follows  (the  other 
category  had  the  same  structure  but  e,  f,  g,  h  occurred  twice  and  a,  b,  c,  d  occurred 
once): 


Person  1  has  features  a  b  e 
Person  2  has  features  a  c  f 
"  3  "  "  dbg 

"  4  "  "  d  c  h 

The  experimental  manipulation  was  that  for  half  the  subjects  Persons  1  and  2  were 
presented  in  red  and  Persons  3  and  4  in  green.  For  the  other  half  of  the  subjects 
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Persons  1  and  3  were  in  red  and  Persons  2  and  4  in  green.  Subjects  knew  that  the  tests 
would  not  include  color.  They  were  shown  each  of  the  8  people  (4  for  the  other 
category)  once  per  block  in  an  anticipation  learning  paradigm  until  a  learning  criterion 
was  met.  A  number  of  different  tests  were  then  given.  The  results  were  quite  clear: 
those  features  correlated  with  color  (e.g.,  a  and  d  for  the  first  half  of  subjects)  were 
viewed  as  more  representative  and  important  to  the  category.  For  instance,  if  a 
correlated  feature  from  one  category  was  presented  with  a  non-correlated  feature  from 
the  other  category,  .74  of  the  categorizations  favored  the  correlated  feature.  Note  that 
in  all  cases,  the  four  critical  features  (a,  b,  c,  d)  occurred  equally  often  with  the  category 
(and  half  as  often  with  the  other  category).  Thus,  it  appears  that  the  reminding  features 
of  color  were  used  to  in  some  way  "organize"  the  category  and  notice  common  features 
within  each  part.  Last  year,  we  replicated  this  study  with  a  number  of  procedural 
changes,  to  insure  its  generality. 

3b.  Remindings  serve  to  focus  the  learner.  One  hypothesis  that  has  not  been 
tested  in  the  formal  domains  is  that  remindings  help  novices  to  focus  on  relevant 
features.  That  is,  one  difficulty  novices  have  is  that  they  are  not  sure  what  information 
is  relevant.  If  a  novice  is  reminded  of  an  appropriate  earlier  problem,  however,  the 
commonalities  are  much  more  likely  to  be  relevant  (compared  to  the  non-common 
aspects)  and  thus  the  novice  may  tend  to  focus  more  on  these  common  aspects.  The 
category  paradigm  presents  a  nice  opportunity  to  examine  this  hypothesis.  In  this  study, 
40  subjects  were  presented  three  times  with  9  members  of  a  single  category  and  told  to 
try  to  learn  about  the  category  for  a  future  test.  As  in  the  above  experiment,  color  was 
correlated  with  some  feature.  In  this  study,  it  was  correlated  with  the  value  of  the  first 
dimension  or  the  second  dimension.  Each  dimension  had  three  values  (e.g.,  hobby  - 
painting,  photography,  weaving)  and  the  three  colors  were  presented  with  the  same 
value  each  time.  In  addition,  half  the  subjects  learned  instances  with  only  two 
dimensions,  while  half  learned  instances  with  four  dimensions  (the  values  for  the  third 
and  fourth  dimensions  were  uncorrelated  with  other  dimensions).  The  idea  was  that  the 
two  dimension  case  was  simple  enough  to  learn  completely,  so  any  focussing  effect  of 
reminding  was  unnecessary.  However,  the  four-dimension  case  was  much  more 
complicated  and  remindings  would  help  learners  to  focus.  For  the  test,  subjects  ranked 
how  important  the  three  values  of  dimension  1  and  the  three  values  of  dimension  2  were 
(i.e.,  the  6  values  were  ranked  1  to  6).  The  results  confirmed  these  predictions.  In  the 
two-dimension  condition,  those  values  correlated  with  color  led  to  only  a  .1  rank 
difference  (out  of  a  possible  3).  In  the  four-dimension  condition,  the  difference  was  1.2, 
with  18  of  20  subjects  showing  the  effect.  The  interaction  with  the  two-dimension 
condition  was  reliable  as  well. 

We  have  conducted  several  follow-ups  to  these  studies  last  year.  Most  important 
were  three  results  using  the  one  category  paradigm.  First  we  varied  the  frequencies  (9, 
12,  or  15  times)  of  presented  features,  for  both  focussed  and  non-focussed.  After  study, 
subjects  were  presented  with  each  feature  and  asked  to  judge  its  frequency.  There  were 
significant  effects  for  frequency  and  for  focussed  vs.  non-focussed.  Focussed  features 
were  rated  as  more  frequent  at  each  level  of  presented  frequency.  This  experiment 
shows  that  the  effect  is  not  due  to  the  forced  choice  nature  of  earlier  studies.  In 
addition,  frequency  effects  are  often  thought  to  relate  closely  to  many  basic  memory 


effects,  suggesting  that  the  manipulation  could  have  a  number  of  implications  for  how 
the  features  are  remembered  and  used. 

Second,  we  conducted  a  more  straightforward  prediction  experiment.  In  this 
study,  each  "person"  had  four  dimensional  features.  In  addition,  the  values  on 
dimensions  1  and  3,  as  well  as  on  2  and  4,  were  perfectly  correlated.  The  color  was 
consistent  with  one  or  the  other  pair.  At  test,  we  would  present,  with  no  color,  two  of 
the  dimensions  filled  in  (e.g.,  the  values  for  1  and  2)  and  the  subjects  had  to  fill  in  the 
values  for  the  other  two  dimensions  (3  and  4,  in  this  instance).  Consistent  with  the 
earlier  findings,  subjects  were  much  better  able  to  fill  in  the  values  consistent  wun  the 
focussed  features,  suggesting  that  they  picked  up  the  correlation  for  those  feature  pairs 
that  were  consistently  colored  (with  an  average  difference  of  4.46  out  of  a  maximum  of 
24). 


Third,  and  just  finished,  we  tested  the  generality  of  the  idea  by  seeing  if  some 
salient  feature,  other  than  color  (which  is  qualitatively  distinct),  would  lead  to  the  same 
effect.  To  test  this,  we  ran  another  experiment  like  part  of  the  focussed  experiment,  but 
used  the  similarity  of  names  rather  than  color  to  be  the  reminding  feature.  Although 
the  effect  was  not  quite  as  large,  we  found  an  advantage  of  .686  out  of  3  maximum,  with 
t(57)  =  2.78. 

These  studies  add  not  only  to  the  idea  of  reminding-based  learning,  but  also  to 
how  categories  might  be  learned.  Future  studies  will  extend  these  findings  with  an 
emphasis  (as  in  the  proposal)  on  the  effects  of  knowledge. 

General  Summary 

I  have  presented  a  brief  summary  of  the  results  from  the  projects  addressing  the 
three  objectives.  Given  that  I  wrote  an  annual  report  in  August  1991,  the  general  thrust 
of  that  report  continues.  We  will  continue  to  concentrate  on  the  irrelevant 
distinctiveness  idea,  problem  categorization,  the  algebra  word  problem  work,  and  the 
Spalding  work  on  concept  formation. 
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